<!DOCTYPE html>
<html lang="zh_cn">
<head>
          <title>来玩魔王的咚</title>
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <meta charset="utf-8" />
        <!-- twitter card metadata -->
<meta name="twitter:card" content="summary">
<meta name="twitter:image" content="/images/mowang.png">
<meta name="twitter:site" content="">
<meta name="twitter:title" content="Python pandas 库">
<meta name="twitter:description" content="<p>简介：安装，导入模块，数据结构，准备数据；DataFrame：生成表格，显示，筛选，排序，索引；高级用法：访问数据，转为数组，统计；进阶操作：map，applymap，apply，聚合，删除；绘图：线性图，np.cumsum累加值，柱状图，直方图，密度图，标准正态分布</p>">
        <!-- OG Tags -->
<meta property="og:url" content="/python-moudle-pandas.html"/>
<meta property="og:title" content="来玩魔王的咚 | Python pandas 库" />
<meta property="og:description" content="<p>简介：安装，导入模块，数据结构，准备数据；DataFrame：生成表格，显示，筛选，排序，索引；高级用法：访问数据，转为数组，统计；进阶操作：map，applymap，apply，聚合，删除；绘图：线性图，np.cumsum累加值，柱状图，直方图，密度图，标准正态分布</p>" />
        <!-- favicon -->
        <link rel="icon" type="image/png" href="/images/mowang.png">
        <!-- moment.js for date formatting -->
        <script src="/theme/js/moment.js"></script>
        <!-- css -->
        <link rel="stylesheet" type="text/css" href="/theme/css/main.css" />
        <!-- 左边的menu，如果页面高度不够，就跟着滚动，否则文章分类显示不全 -->
        <link rel="stylesheet" type="text/css" href="/theme/css/mycss/menu.css" />
		<script>
			
                /*! grunt-grunticon Stylesheet Loader - v2.1.2 | https://github.com/filamentgroup/grunticon | (c) 2015 Scott Jehl, Filament Group, Inc. | MIT license. */
    
    (function(e){function t(t,n,r,o){"use strict";function a(){for(var e,n=0;u.length>n;n++)u[n].href&&u[n].href.indexOf(t)>-1&&(e=!0);e?i.media=r||"all":setTimeout(a)}var i=e.document.createElement("link"),l=n||e.document.getElementsByTagName("script")[0],u=e.document.styleSheets;return i.rel="stylesheet",i.href=t,i.media="only x",i.onload=o||null,l.parentNode.insertBefore(i,l),a(),i}var n=function(r,o){"use strict";if(r&&3===r.length){var a=e.navigator,i=e.Image,l=!(!document.createElementNS||!document.createElementNS("http://www.w3.org/2000/svg","svg").createSVGRect||!document.implementation.hasFeature("http://www.w3.org/TR/SVG11/feature#Image","1.1")||e.opera&&-1===a.userAgent.indexOf("Chrome")||-1!==a.userAgent.indexOf("Series40")),u=new i;u.onerror=function(){n.method="png",n.href=r[2],t(r[2])},u.onload=function(){var e=1===u.width&&1===u.height,a=r[e&&l?0:e?1:2];n.method=e&&l?"svg":e?"datapng":"png",n.href=a,t(a,null,null,o)},u.src="",document.documentElement.className+=" grunticon"}};n.loadCSS=t,e.grunticon=n})(this);(function(e,t){"use strict";var n=t.document,r="grunticon:",o=function(e){if(n.attachEvent?"complete"===n.readyState:"loading"!==n.readyState)e();else{var t=!1;n.addEventListener("readystatechange",function(){t||(t=!0,e())},!1)}},a=function(e){return t.document.querySelector('link[href$="'+e+'"]')},c=function(e){var t,n,o,a,c,i,u={};if(t=e.sheet,!t)return u;n=t.cssRules?t.cssRules:t.rules;for(var l=0;n.length>l;l++)o=n[l].cssText,a=r+n[l].selectorText,c=o.split(");")[0].match(/US\-ASCII\,([^"']+)/),c&&c[1]&&(i=decodeURIComponent(c[1]),u[a]=i);return u},i=function(e){var t,o,a;o="data-grunticon-embed";for(var c in e)if(a=c.slice(r.length),t=n.querySelectorAll(a+"["+o+"]"),t.length)for(var i=0;t.length>i;i++)t[i].innerHTML=e[c],t[i].style.backgroundImage="none",t[i].removeAttribute(o);return t},u=function(t){"svg"===e.method&&o(function(){i(c(a(e.href))),"function"==typeof t&&t()})};e.embedIcons=i,e.getCSS=a,e.getIcons=c,e.ready=o,e.svgLoadedCallback=u,e.embedSVG=u})(grunticon,this);
                
                grunticon(["/theme/css/icons.data.svg.css", "/theme/css/icons.data.png.css", "/theme/css/icons.fallback.css"]);
            </script>
        <noscript><link href="/theme/css/icons.fallback.css" rel="stylesheet"></noscript>
        <!-- menu toggle javascript -->
        <script type="text/javascript">
            document.addEventListener("DOMContentLoaded", initMenu);
            
            function initMenu(){
                var menu = document.getElementById("menu");
                var menulink = document.getElementById("menu-link");
                menulink.addEventListener("click", function toggleMenu(){
                        window.event.preventDefault();
                        menulink.classList.toggle('active');
                        menu.classList.toggle('active');              
                    });
            };
        </script>
        <!-- 不蒜子 -->
        <script async src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script>

    <meta name="description" content="<p>简介：安装，导入模块，数据结构，准备数据；DataFrame：生成表格，显示，筛选，排序，索引；高级用法：访问数据，转为数组，统计；进阶操作：map，applymap，apply，聚合，删除；绘图：线性图，np.cumsum累加值，柱状图，直方图，密度图，标准正态分布</p>" />

    <meta name="tags" content="python模块" />
  <!-- 替换部分base的样式，看文章时，要再宽一点，右边有很多空间可以撑开 -->
  <link rel="stylesheet" type="text/css" href="/theme/css/mycss/article.css" />

</head>
<body>
    <div role="banner" id="masthead">
        <header>
            <a href="/"><img src="/images/mowang.png" alt="McManus Logo"></a>
                <h1>来玩魔王的咚@骑士救兵</h1>
            <a href="#menu" id="menu-link">more stuff</a>
            <nav id="menu">
                <ul>
                        <li><a href="/tags">tags</a></li>
                            <li><a href="/category/cloud.html">Cloud</a></li>
                            <li><a href="/category/docker.html">Docker</a></li>
                            <li><a href="/category/go.html">Go</a></li>
                            <li><a href="/category/linux.html">Linux</a></li>
                            <li class="active"><a href="/category/python.html">Python</a></li>
                            <li><a href="/category/xue-xi-bi-ji.html">学习笔记</a></li>
                            <li><a href="/category/yun-wei-zi-dong-hua.html">运维自动化</a></li>
                </ul>
            </nav>
        </header>
    </div>
        <div class="page" role="main">
  <div class="article" role="article">
    <article>
        <footer>
            <a name="top"></a>
            <p>
              <time datetime=" 2021-10-26 18:25:00+08:00">
                <script>document.write(moment('2021-10-26 18:25:00+08:00').format('LL'));</script>
              </time>
              ~
              <time datetime=" 2021-10-26 18:25:00+08:00">
                <script>document.write(moment('2020-10-26 18:25:00+08:00').format('LL'));</script>
              </time>
            </p>
        </footer>
        <header>
          <h2>
            Python pandas 库
          </h2>
        </header>
      <div class="content">
         <div class="toc">
<ul>
<li><a href="#jian-jie">简介</a><ul>
<li><a href="#an-zhuang">安装</a></li>
<li><a href="#dao-ru-mo-kuai">导入模块</a></li>
<li><a href="#shu-ju-jie-gou">数据结构</a></li>
<li><a href="#zhun-bei-ji-chu-shu-ju">准备基础数据</a></li>
</ul>
</li>
<li><a href="#dataframe">DataFrame</a><ul>
<li><a href="#sheng-cheng-biao-ge">生成表格</a></li>
<li><a href="#xian-shi-shu-ju">显示数据</a></li>
<li><a href="#shai-xuan-shu-ju">筛选数据</a></li>
<li><a href="#pai-xu">排序</a></li>
<li><a href="#suo-yin_1">索引</a></li>
</ul>
</li>
<li><a href="#gao-ji-yong-fa">高级用法</a><ul>
<li><a href="#fang-wen-shu-ju">访问数据</a></li>
<li><a href="#zhuan-wei-shu-zu">转为数组</a></li>
<li><a href="#tong-ji">统计</a></li>
</ul>
</li>
<li><a href="#jin-jie-cao-zuo">进阶操作</a><ul>
<li><a href="#map">map</a></li>
<li><a href="#applymap">applymap</a></li>
<li><a href="#apply">apply</a></li>
<li><a href="#zhuan-zhi">转置</a></li>
<li><a href="#shan-chu">删除</a></li>
</ul>
</li>
<li><a href="#guan-cha-chu-li-shu-ju">观察处理数据</a><ul>
<li><a href="#info">info</a></li>
<li><a href="#describe">describe</a></li>
<li><a href="#tong-ji-zhong-wei-shu">统计中位数</a></li>
<li><a href="#kong-zhi-tong-ji">空值统计</a></li>
<li><a href="#tian-chong-kong-zhi">填充空值</a></li>
</ul>
</li>
<li><a href="#hui-tu">绘图</a><ul>
<li><a href="#zai-jupyter-notebook-zhong-hui-tu">在 jupyter notebook 中绘图</a></li>
<li><a href="#xian-xing-tu">线性图</a></li>
<li><a href="#zhu-zhuang-tu">柱状图</a></li>
<li><a href="#zhi-fang-tu">直方图</a></li>
<li><a href="#mi-du-tu">密度图</a></li>
</ul>
</li>
</ul>
</div>
<h3 id="jian-jie"><a class="toclink" href="#jian-jie">简介</a></h3>
<p>Pandas提供了高性能，易于使用的数据结构和数据分析工具。Pandas用于广泛的领域，包括金融，经济，统计，分析等学术和商业领域。<br>
官方网站：<a href="http://pandas.pydata.org/">http://pandas.pydata.org/</a><br>
查阅用法和参数的话，直接去看API reference：<a href="https://pandas.pydata.org/docs/reference/index.html">https://pandas.pydata.org/docs/reference/index.html</a></p>
<h4 id="an-zhuang"><a class="toclink" href="#an-zhuang">安装</a></h4>
<p>pip 安装：</p>
<div class="highlight"><pre><span></span><code>pip install pandas -i https://mirrors.aliyun.com/pypi/simple/
</code></pre></div>

<h4 id="dao-ru-mo-kuai"><a class="toclink" href="#dao-ru-mo-kuai">导入模块</a></h4>
<p>导入后取个别名方便调用：</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
</code></pre></div>

<h4 id="shu-ju-jie-gou"><a class="toclink" href="#shu-ju-jie-gou">数据结构</a></h4>
<p>Pandas处理以下三种数据结构：</p>
<ol>
<li>系列 Series </li>
<li>数据帧 DataFrame</li>
<li>面板 Panel</li>
</ol>
<p>Series是1维的。DataFrame是2维的，是Series的容器。Panel是3维的，是DataFrame的容器。<br>
<em>Panel是旧版的概念，现在用MultiIndex(多重索引)这个概念。</em><br>
这些数据结构构建在Numpy数组之上，这意味着它们很快。<br>
一般用的最多的就是表格，二维数据，下面只讲了DataFrame。  </p>
<h4 id="zhun-bei-ji-chu-shu-ju"><a class="toclink" href="#zhun-bei-ji-chu-shu-ju">准备基础数据</a></h4>
<p>这里有随机函数生成了一个学生成绩的数据表：</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">random</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[(</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span>
         <span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">((</span><span class="s2">&quot;一班&quot;</span><span class="p">,</span> <span class="s2">&quot;二班&quot;</span><span class="p">,</span> <span class="s2">&quot;三班&quot;</span><span class="p">,</span> <span class="s2">&quot;四班&quot;</span><span class="p">,</span> <span class="s2">&quot;五班&quot;</span><span class="p">)),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">((</span><span class="s2">&quot;男&quot;</span><span class="p">,</span> <span class="s2">&quot;女&quot;</span><span class="p">)),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">),</span>
         <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">100</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">)]</span>
</code></pre></div>

<h3 id="dataframe"><a class="toclink" href="#dataframe">DataFrame</a></h3>
<p>接下来就是使用上面的这份数据进行演示。<br>
生成DataFrame的参数：<a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html">https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html</a><br>
读取数据文件：<a href="https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html">https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html</a>  </p>
<h4 id="sheng-cheng-biao-ge"><a class="toclink" href="#sheng-cheng-biao-ge">生成表格</a></h4>
<p>生成的data是一个二维数组的结果，可以方便的生成表格：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">&quot;序号&quot;</span><span class="p">,</span> <span class="s2">&quot;班级&quot;</span><span class="p">,</span> <span class="s2">&quot;性别&quot;</span><span class="p">,</span> <span class="s2">&quot;语文&quot;</span><span class="p">,</span> <span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">,</span> <span class="s2">&quot;物理&quot;</span><span class="p">,</span> <span class="s2">&quot;化学&quot;</span><span class="p">,</span> <span class="s2">&quot;生物&quot;</span><span class="p">])</span>
</code></pre></div>

<p>这里的参数指定了数据源以及每一行的标题。<br>
如果数据源是csv文件，也提供了专门的方法：</p>
<div class="highlight"><pre><span></span><code><span class="n">csv_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;source.csv&#39;</span><span class="p">)</span>
</code></pre></div>

<p>推荐使用pandas直接读取数据文件，快很多。<br>
另外，如果是压缩的csv文件，不用解压，直接写压缩文件的文件名即可。会根据文件扩展名判断，具体可参考compression参数的说明。  </p>
<p>这里再专注一下生成的df的类型：</p>
<div class="highlight"><pre><span></span><code><span class="nb">print</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">df</span><span class="p">))</span>
<span class="c1"># pandas.core.frame.DataFrame</span>
</code></pre></div>

<p>这个数据类型 DataFrame 是这个模块的核心。  </p>
<h4 id="xian-shi-shu-ju"><a class="toclink" href="#xian-shi-shu-ju">显示数据</a></h4>
<p>下面是显示数据的几个常用示例。  </p>
<h5 id="xuan-zhong-qian-hou-ji-tiao"><a class="toclink" href="#xuan-zhong-qian-hou-ji-tiao">选中 前/后 几条</a></h5>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="n">df</span><span class="o">.</span><span class="n">tail</span><span class="p">()</span>
</code></pre></div>

<p>直接显示df会比较多，默认是5条，可以通过参数指定具体的数量</p>
<h5 id="lie-ming"><a class="toclink" href="#lie-ming">列名</a></h5>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">columns</span>
<span class="c1"># Index([&#39;序号&#39;, &#39;班级&#39;, &#39;性别&#39;, &#39;语文&#39;, &#39;数学&#39;, &#39;英语&#39;, &#39;物理&#39;, &#39;化学&#39;, &#39;生物&#39;], dtype=&#39;object&#39;)</span>
</code></pre></div>

<h5 id="suo-yin"><a class="toclink" href="#suo-yin">索引</a></h5>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">index</span>
<span class="c1"># RangeIndex(start=0, stop=100, step=1)</span>
</code></pre></div>

<h4 id="shai-xuan-shu-ju"><a class="toclink" href="#shai-xuan-shu-ju">筛选数据</a></h4>
<p>下面是筛选数据的常用示例。</p>
<h5 id="shai-xuan"><a class="toclink" href="#shai-xuan">筛选</a></h5>
<p>筛选数学成绩：</p>
<div class="highlight"><pre><span></span><code><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">数学</span> <span class="o">&gt;</span> <span class="mi">88</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">0    False</span>
<span class="sd">1    False</span>
<span class="sd">2    False</span>
<span class="sd">3     True</span>
<span class="sd">4    False</span>
<span class="sd">Name: 数学, dtype: bool</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>这里给的只是布尔值，一般是期望获得一个新的表格，表格里的数据是所有符合条件的数据：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[</span><span class="n">df</span><span class="o">.</span><span class="n">数学</span> <span class="o">&gt;</span> <span class="mi">88</span><span class="p">]</span>
</code></pre></div>

<p>上面的表达式只是计算获得一个新的表格，但是并没有对这个表格进行赋值。df原本的内容是没有改变的，而这个新的表格由于没有引用，之后也不能再使用。<br>
如果需要再用，那就把它传给一个变量。或者赋值给df，更新df里的内容：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">df</span><span class="o">.</span><span class="n">数学</span> <span class="o">&gt;</span> <span class="mi">88</span><span class="p">]</span>
</code></pre></div>

<h5 id="fu-za-de-shai-xuan"><a class="toclink" href="#fu-za-de-shai-xuan">复杂的筛选</a></h5>
<p>也是可以对多个条件进行筛选的，下面是一个例子：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[(</span><span class="n">df</span><span class="o">.</span><span class="n">数学</span> <span class="o">&gt;</span> <span class="mi">88</span><span class="p">)</span> <span class="o">&amp;</span> <span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">语文</span> <span class="o">&lt;</span> <span class="mi">80</span><span class="p">)</span> <span class="o">&amp;</span> <span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">英语</span> <span class="o">&gt;</span> <span class="mi">95</span><span class="p">)]</span>
</code></pre></div>

<h4 id="pai-xu"><a class="toclink" href="#pai-xu">排序</a></h4>
<p>排序，默认是升序：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">([</span><span class="s2">&quot;数学&quot;</span><span class="p">])</span>
</code></pre></div>

<p>降序的话，通过参数指定：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">([</span><span class="s2">&quot;数学&quot;</span><span class="p">],</span> <span class="n">ascending</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</code></pre></div>

<p>另外，还可以对多列进行排序，并且没一列指定是升序或降序：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 第一个降序，第二个升序</span>
<span class="n">df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">([</span><span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">],</span> <span class="n">ascending</span><span class="o">=</span><span class="p">[</span><span class="kc">False</span><span class="p">,</span> <span class="kc">True</span><span class="p">])</span>
</code></pre></div>

<h4 id="suo-yin_1"><a class="toclink" href="#suo-yin_1">索引</a></h4>
<p>按照索引定位：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">序号      1</span>
<span class="sd">班级     五班</span>
<span class="sd">性别      女</span>
<span class="sd">语文    100</span>
<span class="sd">数学     72</span>
<span class="sd">英语     96</span>
<span class="sd">物理     98</span>
<span class="sd">化学     76</span>
<span class="sd">生物     73</span>
<span class="sd">Name: 0, dtype: object</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h5 id="zhi-ding-suo-yin"><a class="toclink" href="#zhi-ding-suo-yin">指定索引</a></h5>
<p>之前没有指定索引，所以使用的是默认生成的索引，也就是0开始的整数。<br>
下面的示例中，会生成一组新的数据，并指定数据的索引：</p>
<div class="highlight"><pre><span></span><code><span class="n">source</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s2">&quot;name&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Adam&quot;</span><span class="p">,</span> <span class="s2">&quot;Barry&quot;</span><span class="p">,</span> <span class="s2">&quot;Clark&quot;</span><span class="p">,</span> <span class="s2">&quot;Diana&quot;</span><span class="p">],</span>
    <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Adam Warlock&quot;</span><span class="p">,</span> <span class="s2">&quot;The Flash&quot;</span><span class="p">,</span> <span class="s2">&quot;Superman&quot;</span><span class="p">,</span> <span class="s2">&quot;Wonder Woman&quot;</span><span class="p">],</span>
    <span class="s2">&quot;universe&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Marvel&quot;</span><span class="p">,</span> <span class="s2">&quot;DC&quot;</span><span class="p">,</span> <span class="s2">&quot;DC&quot;</span><span class="p">,</span> <span class="s2">&quot;DC&quot;</span><span class="p">],</span>
    <span class="s2">&quot;gender&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Male&quot;</span><span class="p">,</span> <span class="s2">&quot;Male&quot;</span><span class="p">,</span> <span class="s2">&quot;Male&quot;</span><span class="p">,</span> <span class="s2">&quot;Female&quot;</span><span class="p">],</span>
<span class="p">}</span>
<span class="n">df2</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="p">(</span><span class="s2">&quot;one&quot;</span><span class="p">,</span> <span class="s2">&quot;two&quot;</span><span class="p">,</span> <span class="s2">&quot;three&quot;</span><span class="p">,</span> <span class="s2">&quot;four&quot;</span><span class="p">))</span>  <span class="c1"># 指定索引</span>
</code></pre></div>

<p>现在这组数据的索引就是index指定的字符串了。</p>
<h5 id="shi-yong-suo-yin-fang-wen"><a class="toclink" href="#shi-yong-suo-yin-fang-wen">使用索引访问</a></h5>
<p>现在依然是使用索引来访问，就不能再用数字了：</p>
<div class="highlight"><pre><span></span><code><span class="n">df2</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="s2">&quot;one&quot;</span><span class="p">]</span>  <span class="c1"># 必须是索引</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">name                Adam</span>
<span class="sd">title       Adam Warlock</span>
<span class="sd">universe          Marvel</span>
<span class="sd">gender              Male</span>
<span class="sd">Name: one, dtype: object</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h5 id="shi-yong-hang-hao-fang-wen"><a class="toclink" href="#shi-yong-hang-hao-fang-wen">使用行号访问</a></h5>
<p>不过有另一个方法，提供使用数字的访问方式：</p>
<div class="highlight"><pre><span></span><code><span class="n">df2</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>  <span class="c1"># iloc 才是真正的行号</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">name                Adam</span>
<span class="sd">title       Adam Warlock</span>
<span class="sd">universe          Marvel</span>
<span class="sd">gender              Male</span>
<span class="sd">Name: one, dtype: object</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h5 id="hun-yong"><a class="toclink" href="#hun-yong">混用</a></h5>
<p>还有一个方法，可以混用上面的任意一种，不过并不推荐使用。使用后也会输出一条 Warning 信息，建议使用 loc 或 iloc：</p>
<div class="highlight"><pre><span></span><code><span class="n">df2</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="s2">&quot;one&quot;</span><span class="p">]</span>  <span class="c1"># ix 可以两种混用，但是不推荐使用，会出现Warning</span>
</code></pre></div>

<h3 id="gao-ji-yong-fa"><a class="toclink" href="#gao-ji-yong-fa">高级用法</a></h3>
<h4 id="fang-wen-shu-ju"><a class="toclink" href="#fang-wen-shu-ju">访问数据</a></h4>
<p>可以使用点的形式访问，类似属性。也可以使用方括号的形式访问，类似字典。<br>
下面的两条效果一样，都是选中数学这一列：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">数学</span>
<span class="n">df</span><span class="p">[</span><span class="s2">&quot;数学&quot;</span><span class="p">]</span>
</code></pre></div>

<h5 id="xuan-ze-duo-lie"><a class="toclink" href="#xuan-ze-duo-lie">选择多列</a></h5>
<p>如果要选择多列，就要用方括号了：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[[</span><span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">]]</span>  <span class="c1"># 内嵌的需要是数组，不能是元组</span>
</code></pre></div>

<p>选中多列的用法，可以只选中一列，注意下面两个语法的差别：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 这是一个 DataFrame，二维的表格，只不过表格仅有1列而已</span>
<span class="n">df</span><span class="p">[[</span><span class="s2">&quot;数学&quot;</span><span class="p">]]</span>

<span class="c1"># 这是一个 Series，一维的</span>
<span class="n">df</span><span class="p">[</span><span class="s2">&quot;数学&quot;</span><span class="p">]</span>
</code></pre></div>

<h5 id="xuan-ze-duo-xing"><a class="toclink" href="#xuan-ze-duo-xing">选择多行</a></h5>
<p>相当于是切片操作：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span>  <span class="c1"># 选择从开头到下标2的那条，一共3条</span>
</code></pre></div>

<h5 id="geng-fu-za-de-xuan-ze"><a class="toclink" href="#geng-fu-za-de-xuan-ze">更复杂的选择</a></h5>
<p>就是上面的方法的联合使用，对行切片并指定需要的列：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[:</span><span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;班级&quot;</span><span class="p">]</span>  <span class="c1"># 只要班级这列</span>
<span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[:</span><span class="mi">2</span><span class="p">,</span> <span class="p">[</span><span class="s2">&quot;语文&quot;</span><span class="p">,</span> <span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">]]</span>  <span class="c1"># 指定更多列</span>
<span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="p">[</span><span class="s2">&quot;语文&quot;</span><span class="p">,</span> <span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">]]</span>  <span class="c1"># 指定需要那几个索引的行</span>
</code></pre></div>

<h5 id="tiao-jian-shai-xuan"><a class="toclink" href="#tiao-jian-shai-xuan">条件筛选</a></h5>
<p>可以使用条件筛选，然后依然是指定加上指定需要的列：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 条件筛选后输出指定列</span>
<span class="n">df</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">df</span><span class="o">.</span><span class="n">班级</span> <span class="o">==</span> <span class="s2">&quot;一班&quot;</span><span class="p">,</span> <span class="p">[</span><span class="s2">&quot;班级&quot;</span><span class="p">,</span> <span class="s2">&quot;性别&quot;</span><span class="p">,</span> <span class="s2">&quot;生物&quot;</span><span class="p">]]</span>
</code></pre></div>

<h4 id="zhuan-wei-shu-zu"><a class="toclink" href="#zhuan-wei-shu-zu">转为数组</a></h4>
<p>使用 values 可以把表格的数组再转为数组：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">values</span>  <span class="c1"># 二维数组</span>
<span class="sd">&quot;&quot;&quot;</span>

<span class="sd">Out[25]:</span>
<span class="sd">array([[1, &#39;五班&#39;, &#39;女&#39;, 100, 72, 96, 98, 76, 73],</span>
<span class="sd">       [2, &#39;一班&#39;, &#39;女&#39;, 92, 84, 93, 70, 92, 99],</span>
<span class="sd">       [3, &#39;二班&#39;, &#39;女&#39;, 81, 72, 76, 80, 98, 77],</span>
<span class="sd">       [4, &#39;四班&#39;, &#39;男&#39;, 99, 89, 89, 86, 85, 86],</span>
<span class="sd">       [5, &#39;五班&#39;, &#39;女&#39;, 93, 85, 82, 85, 92, 72]], dtype=object)</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>这是一个2维数组，如果输出的是某一列，就是1维数组了：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">数学</span><span class="o">.</span><span class="n">values</span>  <span class="c1"># 一维数组</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">array([ 90,  84,  83,  74,  71,  83,  83,  71,  73,  92,  84,  90,  93,</span>
<span class="sd">        98,  91,  81,  86,  84,  83,  84,  97,  83,  73,  96,  81,  98,</span>
<span class="sd">        84,  83,  86,  85,  78,  94,  81,  97,  98,  96,  86,  78,  95,</span>
<span class="sd">        83,  91,  75,  82,  96,  75,  87,  81,  82,  96,  79,  90,  92,</span>
<span class="sd">        80, 100,  77,  81,  79,  80,  75,  87,  71,  97,  97,  98,  98,</span>
<span class="sd">        88, 100,  80,  87,  84,  72,  74,  98,  87,  93,  72,  72,  81,</span>
<span class="sd">        78,  75,  84,  77,  89,  75,  82,  81,  99,  93,  94,  84,  75,</span>
<span class="sd">        78,  92,  88, 100,  70,  99,  92,  81,  76], dtype=int64)</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>维度还是根据数据的类型决定的，这里已经是Series类型了：</p>
<div class="highlight"><pre><span></span><code><span class="nb">type</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">数学</span><span class="p">)</span>
<span class="c1"># pandas.core.series.Series</span>
</code></pre></div>

<h4 id="tong-ji"><a class="toclink" href="#tong-ji">统计</a></h4>
<p>使用 value_counts 可以统计记录的数量：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">班级</span><span class="o">.</span><span class="n">value_counts</span><span class="p">()</span>  <span class="c1"># 统计每个班级的记录数</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">二班    26</span>
<span class="sd">五班    26</span>
<span class="sd">一班    19</span>
<span class="sd">四班    18</span>
<span class="sd">三班    11</span>
<span class="sd">Name: 班级, dtype: int64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h5 id="tong-ji-yi-lie-shu-ju"><a class="toclink" href="#tong-ji-yi-lie-shu-ju">统计一列数据</a></h5>
<p>假设90分以上是优秀，就来看看优秀的数据是怎么样的。<br>
下面这样直接对成绩进行统计效果会很差，因为分数的分布比较多，我们需要统计其中的一段分布，比如这里的90分以上：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">语文</span><span class="o">.</span><span class="n">value_counts</span><span class="p">()</span>
</code></pre></div>

<p>可以在统计之前做筛选，然后再看看各个班级语文优秀的分布：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[</span><span class="n">df</span><span class="o">.</span><span class="n">语文</span> <span class="o">&gt;</span><span class="mi">90</span><span class="p">]</span><span class="o">.</span><span class="n">班级</span><span class="o">.</span><span class="n">value_counts</span><span class="p">()</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">二班    12</span>
<span class="sd">四班     8</span>
<span class="sd">五班     7</span>
<span class="sd">一班     6</span>
<span class="sd">三班     4</span>
<span class="sd">Name: 班级, dtype: int64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>这里做的相当于是聚类，之后讲map函数的时候，也能有类似的效果。先对值进行计算，用返回的新值生成一个新的列，然后再对新列进行统计也是很方便的。</p>
<h5 id="ju-he-ji-suan-zong-shu"><a class="toclink" href="#ju-he-ji-suan-zong-shu">聚合计算总数</a></h5>
<p>简单的聚合，pandas自带的方法就可以实现：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[</span><span class="n">df</span><span class="o">.</span><span class="n">语文</span> <span class="o">&gt;</span><span class="mi">90</span><span class="p">][</span><span class="s2">&quot;语文&quot;</span><span class="p">]</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
<span class="c1"># 37</span>
</code></pre></div>

<p>更多聚类运算就不一一列举了。<br>
复杂的聚合计算可以引入Numpy，这里的DataFrame就是一个二维数组，这个后面会用到。  </p>
<h3 id="jin-jie-cao-zuo"><a class="toclink" href="#jin-jie-cao-zuo">进阶操作</a></h3>
<p>pandas中的dataframe的操作，很大一部分是和numpy中的二维数组的操作是近似的。</p>
<h4 id="map"><a class="toclink" href="#map">map</a></h4>
<p>使用map和自定义函数，可以根据原有的数据，生成新的列：</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">level</span><span class="p">(</span><span class="n">score</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">score</span> <span class="o">&gt;=</span> <span class="mi">90</span><span class="p">:</span>
        <span class="k">return</span> <span class="s2">&quot;优&quot;</span>
    <span class="k">elif</span> <span class="n">score</span> <span class="o">&gt;=</span> <span class="mi">80</span><span class="p">:</span>
        <span class="k">return</span> <span class="s2">&quot;良&quot;</span>
    <span class="k">elif</span> <span class="n">score</span> <span class="o">&gt;=</span> <span class="mi">70</span><span class="p">:</span>
        <span class="k">return</span> <span class="s2">&quot;中&quot;</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">return</span> <span class="s2">&quot;差&quot;</span>
<span class="c1"># 根据原有的数据，使用map和自定义函数，生成新的列</span>
<span class="n">df</span><span class="p">[</span><span class="s2">&quot;数学等级&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">数学</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">level</span><span class="p">)</span>
<span class="n">df</span><span class="p">[[</span><span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;数学等级&quot;</span><span class="p">]]</span>
</code></pre></div>

<h4 id="applymap"><a class="toclink" href="#applymap">applymap</a></h4>
<p>这个函数和上面的map函数有点像，不过是一次对每一个（每一格）数据进行操作。这里做一个简单的例子，把所有的数据，也就是字符串，都用竖线包起来：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">applymap</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s2">&quot;| &quot;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">+</span> <span class="s2">&quot; |&quot;</span><span class="p">)</span>
</code></pre></div>

<h4 id="apply"><a class="toclink" href="#apply">apply</a></h4>
<p>这个函数可以按列（默认）或者按行来操作数据：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># df.apply把每一行，或者每一列的数据传值给x</span>
<span class="c1"># 最终的返回值就是第一个参数func的返回值</span>
<span class="n">df</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">数学</span> <span class="o">+</span> <span class="mf">0.5</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>

<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">0    90.5</span>
<span class="sd">1    84.5</span>
<span class="sd">2    83.5</span>
<span class="sd">3    74.5</span>
<span class="sd">4    71.5</span>
<span class="sd">dtype: float64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>这个例子也不是太好，简单理解一下。匿名函数中x是每一行的数据，后面只取出了 x.数学 这一个值，所以最后输出的就只有数学的值了。</p>
<h5 id="ju-he-ji-suan"><a class="toclink" href="#ju-he-ji-suan">聚合计算</a></h5>
<p>就是计算各种总和、最大、最小、平均值这类，计算使用numpy来实现。下面来计算一个所有人的平均分。<br>
因为取平均只能对数值进行操作，所以需要把对应的列筛选出来才能进行计算：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[[</span><span class="s2">&quot;语文&quot;</span><span class="p">,</span> <span class="s2">&quot;数学&quot;</span><span class="p">,</span> <span class="s2">&quot;英语&quot;</span><span class="p">]]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">average</span><span class="p">)</span>  <span class="c1"># axis默认是0，不写了</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">语文    85.52</span>
<span class="sd">数学    84.90</span>
<span class="sd">英语    84.93</span>
<span class="sd">dtype: float64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p><em>这里只是简单的聚合计算，pandas就有方法可以实现。不过计算方面还是Numpy更强大，需要时可以像这样引入Numpy进行复杂的运算。</em></p>
<h5 id="tong-guo-duo-lie-shu-ju-ji-suan-sheng-cheng-xin-lie"><a class="toclink" href="#tong-guo-duo-lie-shu-ju-ji-suan-sheng-cheng-xin-lie">通过多列数据计算生成新列</a></h5>
<p>下面的例子把物理、化学、生物的成绩综合起来计算出一个综合分：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="p">[</span><span class="s2">&quot;综合&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">round</span><span class="p">((</span><span class="n">x</span><span class="o">.</span><span class="n">物理</span><span class="o">+</span><span class="n">x</span><span class="o">.</span><span class="n">化学</span><span class="o">+</span><span class="n">x</span><span class="o">.</span><span class="n">生物</span><span class="p">)</span> <span class="o">/</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</code></pre></div>

<p>这次把计算的结果赋值给了 df["综合"]，原来的df表格中就多了一个新的列，并且记录了这里计算出的结果。</p>
<h4 id="zhuan-zhi"><a class="toclink" href="#zhuan-zhi">转置</a></h4>
<p>转置可以把行变成列，列变成行。另外原本每行的索引（index）转置后就变成了新的表格的列名（columns），列名则变为索引。<br>
下面单独生成一个小数据，查看转置的效果：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">([[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]],</span> <span class="n">index</span><span class="o">=</span><span class="p">(</span><span class="s2">&quot;奇数&quot;</span><span class="p">,</span> <span class="s2">&quot;偶数&quot;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">T</span>  <span class="c1"># df的转置</span>
</code></pre></div>

<h4 id="shan-chu"><a class="toclink" href="#shan-chu">删除</a></h4>
<p>使用 drop 可以进行删除的操作，下面是一个删除整列的例子，把上面生成的新列删除掉：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="s2">&quot;综合&quot;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># 重新赋值给df</span>
</code></pre></div>

<p>注意这里的参数 axis=1，是指定按列进行操作。<br>
另外这里也用了赋值语句，道理还是要使用操作后的新表格更新原来的df表格。  </p>
<h3 id="guan-cha-chu-li-shu-ju"><a class="toclink" href="#guan-cha-chu-li-shu-ju">观察处理数据</a></h3>
<p>在获取到一组数据后，总是需要大致的看一下这些数据的情况。已经学过 head 和 tail 可以查看最前面和最后的几个数据。另外还有一些好用的函数可以帮助我们对数据有一个大致的了解。</p>
<h4 id="info"><a class="toclink" href="#info">info</a></h4>
<p>info 函数可以获取到所有数据的结构，相当于数据库的表结构：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>

<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;</span>
<span class="sd">RangeIndex: 100 entries, 0 to 99</span>
<span class="sd">Data columns (total 9 columns):</span>
<span class="sd">序号    100 non-null int64</span>
<span class="sd">班级    100 non-null object</span>
<span class="sd">性别    100 non-null object</span>
<span class="sd">语文    100 non-null int64</span>
<span class="sd">数学    100 non-null int64</span>
<span class="sd">英语    100 non-null int64</span>
<span class="sd">物理    100 non-null int64</span>
<span class="sd">化学    100 non-null int64</span>
<span class="sd">生物    100 non-null int64</span>
<span class="sd">dtypes: int64(7), object(2)</span>
<span class="sd">memory usage: 7.1+ KB</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h4 id="describe"><a class="toclink" href="#describe">describe</a></h4>
<p>describe 函数可以自动找到那些数值型的数据，并且计算出各种统计值：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span>
</code></pre></div>

<p><img alt="describe" src="/images/python/10_describe.jpg"></p>
<h4 id="tong-ji-zhong-wei-shu"><a class="toclink" href="#tong-ji-zhong-wei-shu">统计中位数</a></h4>
<p>上面的describe输出的各种统计中，就包括中位数，如果要单独获取到每个数值，可以这样：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">median</span><span class="p">()</span>

<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">序号    50.5</span>
<span class="sd">语文    85.0</span>
<span class="sd">数学    85.0</span>
<span class="sd">英语    85.0</span>
<span class="sd">物理    84.5</span>
<span class="sd">化学    82.0</span>
<span class="sd">生物    86.0</span>
<span class="sd">dtype: float64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<h4 id="kong-zhi-tong-ji"><a class="toclink" href="#kong-zhi-tong-ji">空值统计</a></h4>
<p>很多时候，收集到的数据并不会很完整，这里面就会有很多的空值。所以在使用之前，空值处理是很重要的。首先要进行统计空值，对空值的情况要有一个大概的了解。<br>
先制造一些空值：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 制造几个空值</span>
<span class="n">df</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="mi">8</span><span class="p">,</span> <span class="mi">5</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>
</code></pre></div>

<p>查看空值：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">isnull</span><span class="p">()</span><span class="o">.</span><span class="n">head</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
</code></pre></div>

<p>这个方法只是把原本显示的内容，替换为True或False，以表示该单元格是否为空值。数据很大的时候并不好用。<br>
空值统计：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">isnull</span><span class="p">()</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>

<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">序号    0</span>
<span class="sd">班级    0</span>
<span class="sd">性别    0</span>
<span class="sd">语文    0</span>
<span class="sd">数学    0</span>
<span class="sd">英语    6</span>
<span class="sd">物理    0</span>
<span class="sd">化学    0</span>
<span class="sd">生物    0</span>
<span class="sd">dtype: int64</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>再调用一下sum方法，就把空值的数量统计出来了。</p>
<h4 id="tian-chong-kong-zhi"><a class="toclink" href="#tian-chong-kong-zhi">填充空值</a></h4>
<p>对于空值，可以给这些空值填入一些合适的值。下面就是给这些空值全部填入0：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
</code></pre></div>

<p>上面的操作是给所有的单元格都进行填充。如果多个列都有空值，但是不想全部都填入0，可以只对部分数据进行填充。用好之前的数据筛选的方法就好了，比如只填充英语的空值：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">英语</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
</code></pre></div>

<h5 id="tian-chong-bing-geng-xin"><a class="toclink" href="#tian-chong-bing-geng-xin">填充并更新</a></h5>
<p>上面这些操作并不会改变原本的表格。inplace参数可以直接更新到源数据，下面直接对英语的空值填充中位数，并且更新源数据：</p>
<div class="highlight"><pre><span></span><code><span class="c1"># 默认返回新的值，不会修改原来的值</span>
<span class="c1"># 使用inplace参数，没有返回值，直接修改源数据了</span>
<span class="n">df</span><span class="o">.</span><span class="n">英语</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">英语</span><span class="o">.</span><span class="n">median</span><span class="p">(),</span> <span class="n">inplace</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>

<h5 id="zhong-wei-shu"><a class="toclink" href="#zhong-wei-shu">中位数</a></h5>
<p>对空值填充，需要根据情况选择填充合适的值，比如：零值，平均值、中位数。<br>
中位数和平均值的差别，一般情况下这2个值会比较接近。但是如果数据中有个别极大或者级小的值，这些值就会对平均值的结果产生很大的影响，这些值可以被称作噪声。而中位数则可以在一定程度上排除一些噪声的干扰。所以具体要不要进行空值填充，具体填充什么样的值，还要具体情况具体分析。  </p>
<h3 id="hui-tu"><a class="toclink" href="#hui-tu">绘图</a></h3>
<p>绘图有专门的库matplotlib，一般用这个，功能强大。可以通过pandas直接调用。  </p>
<h4 id="zai-jupyter-notebook-zhong-hui-tu"><a class="toclink" href="#zai-jupyter-notebook-zhong-hui-tu">在 jupyter notebook 中绘图</a></h4>
<p>需要运行下面这句，就可以直接把图画出来了：</p>
<div class="highlight"><pre><span></span><code><span class="o">%</span><span class="n">matplotlib</span> <span class="n">inline</span>
</code></pre></div>

<p>需要先安装matplotlib库。  </p>
<h4 id="xian-xing-tu"><a class="toclink" href="#xian-xing-tu">线性图</a></h4>
<p>使用numpy生成随机数，然后用cumsum方法，产生一个累加的结果，这样可以产生一组不断上升的数据。<br>
cumsum方法，也有一个效果相同的同名函数，类似于这个方法的方法表达式。就是数据是作为函数的第一个参数，还是作为方法的接收者。  </p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="n">columns</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;A&#39;</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;C&#39;</span><span class="p">,</span> <span class="s1">&#39;D&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
</code></pre></div>

<p>图形如下：
<img alt="线性图" src="/images/python/10_plot.jpg"></p>
<p>也可以这样只画一根线：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">A</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
</code></pre></div>

<h5 id="cumsum-shuo-ming"><a class="toclink" href="#cumsum-shuo-ming">cumsum 说明</a></h5>
<p>因为这里产生了4组数据，是一个二维数组。cumsum里的参数指定累加的维度，这个参数需要是小于维度的一个正整数或0。对于二维数组，可选值就是0，或1。我们希望是得到4根累加的线，这里用了0。<br>
不好理解的话，可以看下面的例子：</p>
<div class="highlight"><pre><span></span><code><span class="n">l1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">9</span><span class="p">)])</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">array([[1, 2, 3],</span>
<span class="sd">       [4, 5, 6],</span>
<span class="sd">       [7, 8, 9]])</span>
<span class="sd">&quot;&quot;&quot;</span>

<span class="n">np</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="n">l1</span><span class="p">)</span>  <span class="c1"># 默认值是 None，按照一维数据累加了</span>
<span class="c1"># array([ 1,  3,  6, 10, 15, 21, 28, 36, 45], dtype=int32)</span>

<span class="n">np</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="n">l1</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>  <span class="c1"># 纵向的累加</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">array([[ 1,  2,  3],</span>
<span class="sd">       [ 5,  7,  9],</span>
<span class="sd">       [12, 15, 18]], dtype=int32)</span>
<span class="sd">&quot;&quot;&quot;</span>

<span class="n">np</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="n">l1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>  <span class="c1"># 横向的累加</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd">array([[ 1,  3,  6],</span>
<span class="sd">       [ 4,  9, 15],</span>
<span class="sd">       [ 7, 15, 24]], dtype=int32)</span>
<span class="sd">&quot;&quot;&quot;</span>
</code></pre></div>

<p>回到上面生成的数据，这里生成了4组100个数据，需要纵向的累加。</p>
<h4 id="zhu-zhuang-tu"><a class="toclink" href="#zhu-zhuang-tu">柱状图</a></h4>
<p>首先，还是生成数据。假设有4个人，统计每人每月处理的事件数量（随机生成1到100的整数），如下：</p>
<div class="highlight"><pre><span></span><code><span class="n">case_num</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">4</span><span class="p">))</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">case_num</span><span class="p">,</span>
                  <span class="n">columns</span><span class="o">=</span><span class="p">(</span><span class="s2">&quot;Adam&quot;</span><span class="p">,</span> <span class="s2">&quot;Bob&quot;</span><span class="p">,</span> <span class="s2">&quot;Clark&quot;</span><span class="p">,</span> <span class="s2">&quot;Dan&quot;</span><span class="p">),</span>
                  <span class="n">index</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;Jan&#39;</span><span class="p">,</span> <span class="s1">&#39;Feb&#39;</span><span class="p">,</span> <span class="s1">&#39;Mar&#39;</span><span class="p">,</span> <span class="s1">&#39;Apr&#39;</span><span class="p">,</span> <span class="s1">&#39;May&#39;</span><span class="p">,</span> <span class="s1">&#39;Jun&#39;</span><span class="p">,</span> <span class="s1">&#39;Jul&#39;</span><span class="p">,</span> <span class="s1">&#39;Aug&#39;</span><span class="p">,</span> <span class="s1">&#39;Sep&#39;</span><span class="p">,</span> <span class="s1">&#39;Oct&#39;</span><span class="p">,</span> <span class="s1">&#39;Nov&#39;</span><span class="p">,</span> <span class="s1">&#39;Dec&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="o">.</span><span class="n">bar</span><span class="p">()</span>
</code></pre></div>

<p>展示图形如下：
<img alt="柱状图" src="/images/python/10_plot_bar.jpg"></p>
<p>这里还有一种调用方法，效果是一样的。顺便上面的图有点小，再加个参数设置下尺寸：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;bar&#39;</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">20</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
</code></pre></div>

<p>展示图形如下：
<img alt="柱状图2" src="/images/python/10_plot_bar2.jpg"></p>
<h5 id="lei-jia-de-zhu-zhuang-tu"><a class="toclink" href="#lei-jia-de-zhu-zhuang-tu">累加的柱状图</a></h5>
<p>如果想要查看每月总的事件数量，可以这样：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;bar&#39;</span><span class="p">,</span> <span class="n">stacked</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>  <span class="c1"># 累加的柱状图</span>
</code></pre></div>

<p>展示图形如下：
<img alt="累加柱状图" src="/images/python/10_plot_bar_stacked.jpg"></p>
<h4 id="zhi-fang-tu"><a class="toclink" href="#zhi-fang-tu">直方图</a></h4>
<blockquote>
<p>直方图(Histogram)，又称质量分布图，是一种统计报告图，由一系列高度不等的纵向条纹或线段表示数据分布的情况。 一般用横轴表示数据类型，纵轴表示分布情况。  </p>
</blockquote>
<p>生成4组100个随机数样本：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="n">columns</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;A&#39;</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;C&#39;</span><span class="p">,</span> <span class="s1">&#39;D&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">15</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
</code></pre></div>

<p>生成图形如下：
<img alt="直方图" src="/images/python/10_hist.jpg"></p>
<p>这里的随机样本还是比较少的，如果样本足够多，生成的图形会更加趋于平坦：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">10000</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="n">columns</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;A&#39;</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;C&#39;</span><span class="p">,</span> <span class="s1">&#39;D&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">15</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
</code></pre></div>

<p>增加样本数量后的图形：
<img alt="直方图2" src="/images/python/10_hist2.jpg"></p>
<h4 id="mi-du-tu"><a class="toclink" href="#mi-du-tu">密度图</a></h4>
<p>绘制密度图，需要一个额外的库，所以需要安装一下：</p>
<div class="highlight"><pre><span></span><code>pip install scipy -i https://mirrors.aliyun.com/pypi/simple/
</code></pre></div>

<p>这个库比较大，几十兆，最好指定国内镜像源。  </p>
<p>数据方法，这里生成一组标准正态分布的随机数据：</p>
<div class="highlight"><pre><span></span><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="n">columns</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;A&#39;</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">,</span> <span class="s1">&#39;C&#39;</span><span class="p">,</span> <span class="s1">&#39;D&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="o">.</span><span class="n">kde</span><span class="p">()</span>
</code></pre></div>

<p>图形展示：
<img alt="密度图" src="/images/python/10_plot_kde.jpg"></p>
<blockquote>
<p>标准正态分布俗称高斯分布，正态分布是大自然中最常见的分布，标准正态分布就是期望为0，方差为1的正态分布。  </p>
</blockquote>
<p>要获得其他正态分布，可以先用乘法改变方差，再用加法调整期望值：</p>
<div class="highlight"><pre><span></span><code><span class="mf">2.5</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="o">+</span> <span class="mi">3</span>
</code></pre></div>
      </div>
      <div class="back-to-top">
        <a href="/">HOME</a>
        <a href="#top">TOP</a>
      </div>
    </article>
  </div>
<!-- end article -->
<!-- 页面往下滚动一段之后才会显示TOC -->
<script>
  window.onscroll = function() {
    var tocbox = document.getElementsByClassName('toc')[0];
    var osTop = document.documentElement.scrollTop || document.body.scrollTop;
    var osWidth = document.documentElement.scrollWidth || document.body.scrollWidth;
    // console.log(osTop)
    if (osTop>300 && osWidth>865) {
      tocbox.style.display = "block"
    }
    if (osTop<300 || osWidth<865) {
      tocbox.style.display = "none"
    }
  }
</script>
                <footer>
                    <div class="icons">
                    </div>
                    <span id="busuanzi_container_page_pv" style="padding: 10px">本文阅读量<span id="busuanzi_value_page_pv"></span>次</span>
                    <span id="busuanzi_container_site_pv" style="padding: 10px">本站总访问量<span id="busuanzi_value_site_pv"></span>次</span>
                    <span id="busuanzi_container_site_uv" style="padding: 10px">本站总访客数<span id="busuanzi_value_site_uv"></span>人</span>
                    <p>© <script>document.write(moment().format('YYYY'));</script> 749B</p>
                </footer>
        </div>
</body>
</html>